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closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
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DETAILED ACTION 
Response to Amendment 

1 . The amendment filed 22 October 2007 has been entered and made of record. 

2. The examiner acknowledges the amendments to the specification and the 
replacement Figure 2. 

Response to Arguments 

3. Due to the amendment of Figure 2 and the amendment of paragraph 57 of the 
specification, the objections of Figures 2 and 12 are withdrawn. 

4. The typographical error in paragraph 35 of the specification has been corrected; 
therefore the objection of paragraph 35 is withdrawn. The objection of the abstract 
remains. 

5. Applicant's arguments with respect to independent claims 1, 4, 7, 9, 12, and 15 
have been considered but are moot in view of the new ground(s) of rejection. 

Specification 

6. The abstract of the disclosure is objected to because the first sentence of the 
abstract is not a complete sentence. Correction is required. See MPEP § 608.01(b). 

The abstract should be in narrative form and generally limited to a single 
paragraph on a separate sheet within the range of 50 to 150 words. It is important that 
the abstract not exceed 150 words in length since the space provided for the abstract 
on the computer tape used by the printer is limited. The form and legal phraseology 
often used in patent claims, such as "means" and "said," should be avoided. The 
abstract should describe the disclosure sufficiently to assist readers in deciding whether 
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there is a need for consulting the full patent text for details. The language should be 
clear and concise and should not repeat information given in the title. It should avoid 
using phrases which can be implied, such as, "The disclosure concerns," "The 
disclosure defined by this invention," "The disclosure describes," etc. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1-16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

Martins (U.S. Patent 6,608,615) and Edwards (U.S. Patent 6,106,1 19) in view of 

Matraszek et al. (U.S. Patent Publication 2003/0122839). 

With respect to claims 1 and 4, Martins discloses web browsing applications 

that track a user's eye gaze while the user is browsing a web page and modifies the 

presentation of the web page to the user based on the tracked gaze. Martins give the 

example that sections of a web page that a user has previously viewed may be 

represented in a progressively changed form, such as in a different color, brightness, or 

contrast based on the user's tracked gaze. Modifying the color, brightness, or contrast is 

analogous to dynamically acting on a characteristic of the application. The display of the 

web page is also the output of the application, therefore modifying how the web page is 

displayed is analogous to dynamically acting on an output of the application (column 2, 

lines 28-50). 
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Martins does not specifically describe that interpretations of eye tracking data is 
received from an eye interpretation engine. 

Edwards discloses a system and method for presenting high-level interpolations 
of eye tracking data correlated to saved display images (column 1, lines 15-19). 
Edwards describes an eye interpretation engine (EIE) that performs a three-level 
interpretation process. Level one processing analyzes the raw eye-tracking data to 
identify elementary features, typically fixations, saccades, smooth pursuit motion and 
blinks. Level two processing analyzes the elementary features to identify eye-movement 
patterns, typically consisting of a set of several fixations and/or saccades satisfying 
certain predetermined criteria. Level three processing analyzes the eye-movement 
patterns to identify various eye-behavior patterns (interpretations of eye tracking data) 
(column 7, line 5 - column 8, line 21). Next Edwards describe that a graphical valuation 
vocabulary (GW) is assigned to the interpretations of all the levels (column 8, lines 31- 
41), the GW is superimposed on the correlated display scenarios (column 8, lines 42- 
51), and the correlated results are displayed (column 8, lines 52-57). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to use the eye interpretation engine, as taught by Edwards, to generate 
interpretations of eye tracking data in the web browsing application of Martins, in order 
to provide numerous pieces of information about eye behavior patterns and mental 
states to the gaze tracking processing (Edwards, column 2, lines 19-22). 

Neither Martins, nor Edwards describe the use of external context data. 



Application/Control Number: Page 5 

10/821,800 

Art Unit: 2624 

Matraszek et al. disclose methods for determining affective information related to 
images and using the affective information to retrieve digital images (paragraph 28). 
Examples of affective information (external context data) are viewing time, 
behavioral/usage information, facial expressions, body gestures, eye movements (gaze 
tracking), voice, pressure of a user's hand on an input device, and the user's biometric 
or physiological responses as the user views an image. Types of physiological 
responses are EMG, temperature, Galvonic Skin Response, heart rate, EEG, and brain- 
imaging signals (paragraphs 34-42). The affective information (external context data) is 
stored as metadata associated to an image (paragraph 47). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the presentation of a web page in the web browsing application of 
Martins and Edwards, based on the eye tracking data and affective information (external 
context data), as taught by Matraszek et al., in order to take into consideration the 
user's reaction to a web page when choosing how to modify its presentation (Matraszek 
et al., paragraph 12). 

With respect to claims 2 and 5, Martins describes that sections of a web page 
that a user has previously viewed may be represented in a progressively changed form, 
such as in a different color, brightness, or contrast based on the user's tracked gaze. 
Modifying the color, brightness, or contrast is analogous to dynamically modifying a 
characteristic of the application. The display of the web page is also the output of the 
application; therefore modifying how the web page is displayed is analogous to 
dynamically modifying an output of the application (column 2, lines 28-50). 
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With respect to claims 3 and 6, Martins further describe, in another example, 
that an information browsing application may be used to selectively remind air traffic 
controllers of eventful regions of the display screen not being looked at (portion of an 
interface that has not been viewed by the user). The color, brightness, or contrast of the 
ignored regions of the display screen may be modified to alert air traffic controller to 
view the region (modifying a format of the portion of the interface that has not been 
viewed by the user) (column 6, lines 2-4). 

With respect to claim 7, Martins discloses web browsing applications that track 
a user's eye gaze while the user is browsing a web page and modifies the presentation 
of the web page to the user based on the tracked gaze. Martins give the example that 
sections of a web page that a user has previously viewed may be represented in a 
progressively changed form, such as in a different color, brightness, or contrast based 
on the user's tracked gaze (column 2, lines 28-50). 

Martins does not specifically describe that interpretations of eye tracking data is 
received from an eye interpretation engine. 

Edwards discloses a system and method for presenting high-level interpolations 
of eye tracking data correlated to saved display images (column 1, lines 15-19). 
Edwards describes an eye interpretation engine (EIE) that performs a three-level 
interpretation process. Level one processing analyzes the raw eye-tracking data to 
identify elementary features, typically fixations, saccades, smooth pursuit motion and 
blinks. Level two processing analyzes the elementary features to identify eye-movement 
patterns, typically consisting of a set of several fixations and/or saccades satisfying 
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certain predetermined criteria. Level three processing analyzes the eye-movement 
patterns to identify various eye-behavior patterns (interpretations of eye tracking data) 
(column 7, line 5 - column 8, line 21). Next Edwards describe that a graphical valuation 
vocabulary (GW) is assigned to the interpretations of all the levels (column 8, lines 31- 
41), the GW is superimposed on the correlated display scenarios (column 8, lines 42- 
51), and the correlated results are displayed (column 8, lines 52-57). Edwards describes 
that every time a predetermined eye behavior pattern of the test person is recognized 
by the software program, a significant display event takes place and a snapshot is 
recorded. In the combination of Martins and Edwards, the web page is dynamically 
acted on when interpretations of the eye tracking data is received (Edwards, column 9, 
lines 20-34). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to use the eye interpretation engine, as taught by Edwards, to generate 
interpretations of eye tracking data in the web browsing application of Martins, in order 
to provide numerous pieces of information about eye behavior patterns and mental 
states to the gaze tracking processing (Edwards, column 2, lines 19-22). 

Neither Martins, nor Edwards describe the use of external context data. 

Matraszek et al. disclose methods for determining affective information related to 
images and using the affective information to retrieve digital images (paragraph 28). 
Examples of affective information (external context data) are viewing time, 
behavioral/usage information, facial expressions, body gestures, eye movements (gaze 
tracking), voice, pressure of a user's hand on an input device, and the user's biometric 
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or physiological responses as the user views an image. Types of physiological 
responses are EMG, temperature, Galvonic Skin Response, heart rate, EEG, and brain- 
imaging signals (paragraphs 34-42). The affective information (external context data) is 
stored as metadata associated to an image (paragraph 47). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the presentation of a web page in the web browsing application of 
Martins and Edwards, based on the eye tracking data and affective information (external 
context data), as taught by Matraszek et al., in order to take into consideration the 
user's reaction to a web page when choosing how to modify its presentation (Matraszek 
et aL, paragraph 12). 

With respect to claim 8, as discussed above, Martins discloses web browsing 
applications that track a user's eye gaze while the user is browsing a web page and 
modifies the presentation of the web page to the user based on the tracked gaze 
(column 2, lines 28-50). As discussed above, Edwards discloses a system and method 
for presenting high-level interpolations of eye tracking data correlated to saved display 
images (column 1, lines 15-19). Edwards describes that every time a predetermined eye 
behavior pattern of the test person is recognized by the software program, a significant 
display event takes place and a snapshot is recorded. In the combination of Martins and 
Edwards, the update of the web page (application) is dynamically invoked when 
interpretations of the eye tracking data is received (Edwards, column 9, lines 20-34). 

With respect to claims 17-19, as discussed above, Martins discloses web 
browsing applications that track a user's eye gaze while the user is browsing a web 
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page and modifies the presentation of the web page to the user based on the tracked 
gaze (column 2, lines 28-50). As discussed above, Edwards discloses a system and 
method for presenting high-level interpolations of eye tracking data correlated to saved 
display images (column 1, lines 15-19). 

Neither Martins, nor Edwards describe external sources for context data. 

Matraszek et al. disclose methods for determining affective information related to 
images and using the affective information to retrieve digital images (paragraph 28). 
Examples of affective information (external context data) are viewing time, 
behavioral/usage information, facial expressions, body gestures, eye movements (gaze 
tracking), voice, pressure of a user's hand on an input device, and the user's biometric 
or physiological responses as the user views an image. Types of physiological 
responses are EMG, temperature, Galvonic Skin Response, heart rate, EEG, and brain- 
imaging signals (paragraphs 34-42). The affective information (external context data) is 
stored as metadata associated to an image (paragraph'47). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the presentation of a web page in the web browsing application of 
Martins and Edwards, based on the eye tracking data and affective information (external 
context data), as taught by Matraszek et al., in order to take into consideration the 
user's reaction to a web page when choosing how to modify its presentation (Matraszek 
et al., paragraph 12). 

With respect to claims 9-11, 12-14, 15-16, and 20-22, the "article comprising a 
computer-readable medium having stored thereon instructions" corresponds to the 
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methods of claims 1-3, 4-6, 7-8, and 17-19. The argument is the same as is addressed 
above. 

Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Damon Conover whose telephone number is (571) 272- 
5448. The examiner can normally be reached Monday - Friday, 8:30 a.m. - 5:00 p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian Werner, can be reached at (571) 272-7401 . The fax number for the 
organization where this application or proceeding is assigned is (571) 273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at (866) 217-9197 (toll-free). If you would like assistance from a USPTO 
Customer Service Representative or access to the automated information system, call 
(800) 786-9199 (IN USA OR CANADA) or (571) 272-1000. 
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